Adya logo Adya Wisdom
Anatomy of the Harness · Issue 01 of 10

The model was never the problem

Two of the most-cited numbers in enterprise AI say the same thing — and the model isn't one of the reasons. What's actually killing these projects, and the one question that separates a tool you rent from a system you own.

By Shayak Mazumder, Adya · 22 July 2026 · For developers, startup founders, and PMs · ~7 min read
40%+
Agentic AI projects Gartner expects canceled by end of 2027
~95%
Enterprise GenAI pilots MIT found at zero measurable return
$3,600→$360
Same monthly workload, once memory and a tuned model are added
1 question
Is your cost per task falling every week? Flat = tool. Falling = system.
TL;DR
  • The verdict is in and it is brutal. Gartner expects over 40% of agentic AI projects to be canceled by 2027; an MIT study puts ~95% of enterprise GenAI pilots at zero return. Both name the same causes, and the model is not one of them.
  • The killers are cost, governance, and a learning gap — tools that demo well but never learn, never integrate, and never get cheaper. That is an architecture problem, not a model problem.
  • Run the arithmetic: a stateless agent on a rented frontier model can cost ~$3,600 a month on one workload, and it never falls. Add memory and a fine-tuned model and the same work drops toward ~$360.
  • One question separates the survivors from the write-offs: is your cost per task falling every week? Flat = you bought a tool. Falling = you own a system. This issue is where that question comes from; the next nine build it out.
MONTH 1 → MONTH 12 $ / MONTH SAME STARTING COST Rented tool flat · ~$3,600/mo Owned system falling · toward ~$360/mo
Fig 1 · A tool's cost is flat. A system's cost bends down.

Section 01Everyone agrees it's failing. Almost no one agrees why.

Two numbers are in every board deck right now.

Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, on escalating costs, unclear business value, and weak risk controls. And an MIT study of 300 deployments found that roughly 95% of enterprise GenAI pilots deliver zero measurable return, against $30–40 billion spent, with only about 5% reaching production.

Read both post-mortems and the same pattern falls out. Gartner's list of causes is cost, unclear value, and risk. Model capability did not make the cut. MIT is blunter: the divide between the 5% who win and the 95% who stall is not explained by model quality.

The learning gap

They call it a learning gap — the distance between a tool that dazzles in a demo and one that learns, adapts, and survives a real workflow.

Andrej Karpathy has been making the same point to the same audience. His line that this is "the decade of agents, not the year" was a shot at the hype, and his reason is precise: today's agents can't plan, and they can't remember, so their knowledge is pre-compiled instead of compounding. That is not a complaint about the model. It's a description of everything wrapped around it. There's even a name for that wrapper now — Ethan Mollick, Philipp Schmid, and Martin Fowler's team have all spent 2026 writing about the harness. This series takes it apart, one layer at a time.

Section 02The stack isn't the problem. The space between the tools is.

Every second builder call I take opens with the same diagram: n8n for the workflows, Replit or Bolt for the app, LangGraph for orchestration, Airbyte for the pipelines, a memory layer like Honcho on the side. Each tool is good. For a weekend prototype the stack is close to magic. Then production arrives, and the diagram turns into a bill.

Look at what you actually build to take that diagram live. The tools don't share state, so you write the glue for every pair. The memory layer doesn't know your domain, so you design the schema by hand. Nothing evaluates itself, so the evals and observability are yours. Then the permanent job starts: token optimization, caching, context pruning, routing cheap calls to cheap models.

The DIY stack

Five tools, plus the glue between every pair.
  • n8n for workflows, Replit or Bolt for the app, LangGraph for orchestration, Airbyte for pipelines, a memory layer bolted on the side
  • You hand-write the glue for every pair, and hand-design the memory schema
  • Nothing evaluates itself — the evals and observability are yours, forever
  • Cost: flat, forever.

One managed loop

Memory, orchestration, and evals as one system.
  • Shared state across every step — no glue code to write or maintain by hand
  • A domain-aware memory schema, not designed from scratch by hand
  • Evals and observability built in, not bolted on afterward
  • Cost: falls, on purpose.
The integration tax

Paid once in build months, then forever in maintenance. And while you pay it, the meter runs.

Section 03Six cents a task is nothing — until you put it on a workload.

A stateless agent has no memory of its own work, so every call re-sends the whole ritual: system prompt, tool definitions, retrieved documents, history. Fifteen thousand tokens in and a thousand out per task is ordinary for real enterprise work. At frontier list prices near $3 per million in and $15 per million out, that's about six cents a task. On two thousand tasks a day, here's where it lands.

$3,600 $1,080 $360 Stateless + continuous learning + fine-tuned model -70% -90%
Fig 2 · $3,600 → $1,080 → $360 on the same workload.

The first bar should bother you for its shape, not its height. On day 400 the stateless agent pays exactly what it paid on day 1 — same work run 800,000 times, nothing learned from any of it. The second bar is what happens when the system keeps an event-sourced record of what it has done and stops re-shipping context wholesale: roughly 70% fewer tokens over time. The third is the step most teams never reach, because the integration tax ate the roadmap first — fine-tune a small model on your own traces for the lanes you've already solved, and cut that bill by roughly 90%.

This is exactly what MIT found from the other side. The 5% who crossed the divide weren't the ones with the best model. They were the ones who shipped tools with memory and learning loops, and who bought from partners rather than building alone — vendor-built systems reached production about twice as often as internal builds.

On day 400 the stateless agent pays exactly what it paid on day 1 — same work run 800,000 times, nothing learned from any of it.

The series

Keep reading the series

This is Issue 01 of 10. Each one takes a hard problem between a demo and a production agent, and the mechanism that removes it. Forwarded this? Get the next nine in your inbox instead of hoping someone sends them.

Subscribe free

Section 04A tool's cost is a flat line. A system's cost bends down.

Name what you're watching: the COGS curve. Your tokens are cost of goods sold, and over a year the only thing that matters is which way the curve points. Flat means you bought a tool. Falling means you own a system. It's a one-question audit, and it works on any platform, including ours.

Jensen Huang put the same idea more bluntly than I can. At NVIDIA's 2026 GTC, arguing that architecture is everything in the agentic era:

The wrong architecture, even if it's free, is not cheap enough.

Jensen Huang, CEO, NVIDIA — GTC 2026

That's the whole thesis in eleven words. A free model on the wrong architecture still loses, because the cost that sinks you was never the model's price tag. It was the harness — or the absence of one.

Case study · illustrative composite

A mid-market insurer ran a document-triage agent on the standard DIY stack: an orchestration framework, a bolted-on memory store, a frontier model behind every call. It worked in the pilot and stalled on cost the moment volume passed a few thousand claims a day — the exact cliff MIT describes. Moved onto one managed loop with event-sourced memory, the agent began reusing what it had already solved instead of re-paying for it, and the settled document types were routed to a small fine-tuned model. The curve bent inside two months.

  • ~70%fewer tokens per task, once memory compounded
  • ~90%lower cost on the lanes moved to the owned model
  • 0rewrites when the base model was repriced

Illustrative composite representative of the pattern, not one named account.

Section 05The five-letter test: COGSS

Cost is only the first letter of how we grade any AI decision. The whole scorecard is COGSS.

  1. CCostIs the per-task curve falling, or flat forever?
  2. OOwnershipDo your traces and tuned models sit on your side?
  3. GGovernanceAre rules enforced, or only requested in a prompt?
  4. SScaleCan you run 100 agents without a rewrite?
  5. SSovereignIs a model swap a config change, or a crisis?

Read those back and they aren't five features. They're five ways of asking one question: does the system belong to you, or are you renting a capability you don't control? Every issue in this series takes one letter and one hard problem and shows the mechanism that answers it.

Section 06What to do Monday

You don't need our platform to use any of this. Take one agent in production, plot its cost per task for the last eight weeks, and look at the slope. Flat line: you have a tool wearing the costume of a system, and no model upgrade will bend it, because the model was never the variable. Falling line: find what's bending it and do more of that. Everything in the next nine issues is a way to bend that line on purpose.

The one-question audit

Is your cost per task falling every week? Flat = you bought a tool. Falling = you own a system.

Get the next nine

…and actually receive them

  1. Subscribe, free — here, one click, no forms.
  2. So this doesn't land in spam, add shayak@adya.ai to your contacts, and if it's in Promotions, drag it to Primary.
  3. Want to see a falling curve on your own workload? Open SuperAgent AI.
Subscribe to Anatomy of the Harness

I run Adya, where we build agents and test models for a living, so most of what I write in this series is whatever broke on us first — our own bills included. Keep the lens whether or not you ever touch our platform. That's the point of writing it down.

See you in Issue 02, where we get into why your data warehouse full of synced tables still isn't something an agent can actually reason over.

— Shayak · the rough drafts of these ideas go up on my LinkedIn first.

Sources

  1. Gartner, "Over 40% of Agentic AI Projects Will Be Canceled by End of 2027," Jun 2025 — gartner.com
  2. MIT NANDA, "The GenAI Divide: State of AI in Business 2025," Jul 2025 — via Fortune
  3. Andrej Karpathy, "decade of agents / agents can't remember," Dwarkesh Patel interview, Oct 2025.
  4. Jensen Huang, NVIDIA GTC 2026 keynote — via RCR Wireless